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Abstract 

Systematic  has  developed  a debriefing  system  for 
aircraft  crews  to  improve  their  skills  based  on 
experiences  from  completed  missions.  The  system  is 
developed  on  Commercial  Off  The  Shelf  (COTS) 
software  and  on  a PC.  The  panel  should  see  this  input  as 
a portable,  low-cost  Virtual  Reality  (VR)  training  system 
for  aircraft  crews.  The  benefit  of  the  portability  is  that 
the  system  can  be  used  anywhere  the  unit  is  deployed 
and  by  any  crewmember. 

Flight  hours  are  rather  expensive  and  therefore  the  air 
forces  must  maximise  the  benefits  from  spent  flight 
hours.  This,  combined  with  the  fact  that  most  air  force 
units  need  to  operate  from  different  deployments  remote 
from  home  bases,  led  the  operational  fighter  squadrons 
to  express  a need  for  a low-cost  debriefing  system. 

The  users  were  directly  involved  in  the  design  and  the 
focus  was  set  on  functionality  — not  technology.  This 
approach  has  resulted  in  a system  which  gains  accept 
among  users  and  therefore  becomes  an  everyday  training 
tool.  Driven  by  user  requirements,  the  system  is 
developed  to  run  on  a Microsoft  Windows  2000 
platform,  and  the  system  can  interface  with  other 
systems.  Furthermore,  it  has  been  essential  to  develop  a 
system,  which  could  be  rapidly  implemented. 

The  debriefing  system  uses  already  existing  information 
from  the  aircraft.  The  aircraft  is  equipped  with  Global 
Positioning  System  (GPS),  three  video  cameras,  and  a 
microphone  system  to  record  the  pilot’s  voice 
communication.  The  video  cameras  record  the  pilot’s 
view  through  his  head-up  display  and  the  entire 
instrument  panel. 

Prior  to  the  debriefing  session  all  information  from  the 
aircraft  (GPS-data,  video-  and  audio  recordings)  is  fed 
into  the  debriefing  system.  The  GPS-data  is  loaded  into  a 
three  dimensional  (3D)  model  containing  geographical 
information,  the  video  and  audio  recordings  are 
digitised,  and  all  data  are  synchronised.  On  each 
monitor,  four  visual  sources  can  be  displayed 
concurrently,  e.g.  video  recordings  from  three  different 
aircraft  and  the  graphical  3D  view  of  the  area,  including 
aircraft.  The  selected  visual  sources  are  displayed  along 
with  a selected  audio  recording.  The  3D  graphic  makes  it 
possible  to  see  and  follow  selected  aircraft  from  different 
perspectives  on  their  mission.  Furthermore,  it  is  possible 
to  see  them  chase  other  aircraft  and  to  track  their  route 


by  position,  direction,  and  speed.  The  crew  and  other 
mission  participants  can  by  themselves  prepare  and 
execute  the  debriefing  session. 

Systematic  has  developed  a portable,  low-cost  VR 
training  system  for  aircraft  crews,  which  converts  reality 
to  virtual  reality,  reflecting  the  reality.  The  chosen 
approach,  with  heavy  user  involvement,  has  resulted  in  a 
system,  which  is  easy  to  use  and  will  gain  much  better 
acceptance.  A system  based  on  well-proven  COTS 
products  reduces  costs  as  well  as  risks.  Finally,  the 
system  gives  added  value  to  the  flight  hours  spent. 

Introduction 

It  is  our  aim  with  this  paper  to  disseminate 
understanding  for  the  possibilities  given  by  new 
commercial  off  the  shelf  (COTS)  products  — in  this  case 
especially  for  low  cost  virtual  reality  training  tools.  We 
find  that  today’s  COTS  software  fulfils  most  of  the 
requirements  that  the  military  has  to  an  everyday 
debriefing  system.  By  combining  the  COTS  products 
using  Systematic’s  competence  in  software  integration,  a 
low-cost  easy-to-operate  operational  training  system,  has 
been  developed. 

Through  this  paper,  we  discuss  functional  requirements, 
use  of  commercial  state  of  the  art  technology,  influence 
on  training  and  human  performance  requirements,  and 
describe  the  development  process  and  functionality  in 
our  debriefing  system. 

In  connection  with  the  training  of  combat  pilots  much 
time  is  spent  on  manoeuvres  in  actual  air  combat 
techniques.  The  Danish  Air  Force  spends  more  than  half 
of  the  flight  hours  on  such  manoeuvres.  Furthermore,  the 
remaining  flight  hours  often  contains  elements  of  air 
combat.  It  is  therefore  essential  to  get  full  benefit  from 
the  training,  especially  as  flight  hours  are  extremely 
costly.  Nevertheless,  subsequent  debriefing  and  evalua- 
tion of  a training  session  is  often  deficient  or  non- 
existing. 

The  present  project  has  endeavoured  to  remedy  this 
inadequacy  by  investigating  the  possibilities  for  building 
an  inexpensive,  simple  and  user-friendly,  but  yet  high- 
tech,  mission  debriefing  system,  for  “everyday  use”.  We 
have  used  virtual  reality  (VR)  and  3D  techniques  for 
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constructing  factual  conditions  for  training  in  a Virtual 
Environment  (VE).  The  VE  facilitates  the  debriefing  of 
pilots  and  thereby  enhances  the  learning.  Presently  the 
system  is  developed  as  a 1 . generation  version  with  basic 
functionality  financed  by  our  company.  We  find  though, 
that  the  idea  has  much  potential,  and  we  will  promote 
our  ideas  broadly  within  NATO. 

Background 

Security  in  the  Euro-Atlantic  area  has  substantially 
improved  during  the  1990s,  by  comparison  with  the  four 
decades  that  preceded  them.  The  threat  of  massive 
military  confrontation  has  gone,  and  co-operative 
approaches  to  security  have  replaced  former 
confrontation.  Nevertheless  potential  risks  to  security 
from  instability  or  tension  still  exist. 

In  these  changed  circumstances  affecting  Europe’s 
security,  NATO  forces  have  been  adapted  to  the  new 
strategic  environment  and  have  become  smaller  and 
more  flexible.  Conventional  forces  have  been 
substantially  reduced  and  in  most  cases  so  has  their  level 
of  readiness.  They  have  also  been  made  more  mobile,  to 
enable  them  to  react  to  a wider  range  of  contingencies; 
and  they  have  been  reorganised  to  ensure  that  they  have 
the  flexibility  to  contribute  to  crisis  management  and  to 
enable  them  to  be  built  up,  if  necessary,  for  the  purposes 
of  defence.  Increased  emphasis  has  been  given  to  the 
role  of  multinational  forces  within  NATO’s  integrated 
military  structure.  Many  such  measures  have  been 
implemented.  Others  are  being  introduced  as  the  process 
of  adaptation  continues. 

Airforces  are  characterised  by  their  ability  to  operate 
from  far  distance,  geographically  dispersed  bases  and 
concentrate  their  efforts  against  the  main  targets.  They 
are  also  able  to  react  very  fast  and  to  maintain  a high 
degree  of  readiness.  These  characteristics  have  made 
airforces  even  more  important  to  NATO’s  new  strategic 
concept,  Combined  Joint  Task  Forces  (CJTF).  The  main 
issue  in  this  concept  regarding  air  forces  is  high 
readiness,  interoperability  and  the  ability  to  operate  away 
from  home  bases  with  a minimum  of  preparations. 
Furthermore  each  participating  unit  must  be  able  to 
perform  a larger  variety  of  roles,  than  before  — e.g. 
using  heavy  bombers  for  close  air  support.  The 
operational  environment  has  become  much  more 
dynamic  — it  is  never  possible  to  foresee  which  type  of 
operation  that  will  turn  up.  This  again  puts  higher 
demands  on  continuous  and  flexible  training. 

Another  consequence  of  the  new  operational 
environment  is  the  reduced  military  budgets.  This  means 
that  it  is  essential  to  gain  as  much  as  possible  from  the 
applied  training  efforts.  In  real  operations  like  Allied 
Force  in  Kosovo  last  year,  it  is  extremely  important  that 
the  pilots  learn  from  each  mission  to  make  continues 
improvements.  In  this  specific  example,  most  of  the 
participating  units  operated  far  away  from  home  bases. 


Therefore  they  were  not  able  to  take  advantage  of  their 
usual  static  training  equipment  and  simulators. 

Project  Objectives  and  Means 

An  obvious  need  for  mobile  training,  rehearsal  and 
debriefing  systems  has  evolved.  Given  the  fast 
development  within  virtual  reality  technology  and  low 
cost  flight  simulators  for  PCs,  we  have  seen  a good 
opportunity  to  use  commercial  technology  and  existing 
sensors,  video  recordings,  and  tapes  from  the  aircraft  to 
develop  a debriefing  system  for  air  force  pilots. 

The  overall  objective  was  to  create  a low  cost,  easy-to- 
operate,  and  transportable  debriefing  system.  With  this 
objective  the  intention  was  that  each  training  session  and 
live  mission  should  be  followed  up  by  a high-quality 
debriefing  activity,  giving  full  benefit  of  the  costly  flying 
time  to  the  pilots. 

The  aim  was  to  base  the  system  on  COTS  products  and 
existing  and  electronically  available  data  from  the 
aircraft.  Furthermore,  the  aim  was  to  combine  the 
collected  data  from  the  aircraft  and  thereby  constitute  a 
3D  Virtual  Reality  (VR)  replay  of  the  completed 
missions  and  training  sessions. 

The  project  is  financed  by  Systematic  and  the  ingredients 
used  are  Systematic’s  skills  and  knowledge, 
technologically  as  well  as  military,  a range  of  COTS 
products,  and  the  requirements  set  by  airforce  pilots. 

System  requirements  and  functionality 

This  section  describes  the  scenarios  and  missions  that  are 
supported  by  the  debriefing  system.  To  stress  out  the 
need  for  a debriefing  system,  we  give  a brief  description 
of  the  main  categories  of  existing  systems. 

Air  Combat  Manoeuvring  (ACM) 

Air  combat  comprises  all  kinds  of  manoeu  vres  in  the  air 
in  a one-to-one,  many-to-one  or  many-to-many  situation. 
ACM  includes  all  the  classical  movement  patterns  such 
as  half  loop,  full  loop,  split  S,  break  turn  etc.  Basically, 
air  combat  is  a question  of  gaining  the  right  position  in 
relation  to  the  opponent. 

A training  session  consists  of  a number  of  scenarios, 
ranging  from  3 to  10  — depending  on  the  number  of 
fighters  involved.  A scenario  lasts  from  5 to  10  minutes. 
The  starting  point  of  a scenario  is  an  initial  position 
where,  for  example,  the  different  players  have  got  radar 
contact  (approx.  30NM  distance).  Typically,  the 
situation  then  develops  rapidly,  depending  on  the  actions 
that  take  place  during  the  session.  After  only  a few 
minutes,  the  situation  typically  becomes  very  complex 
and  the  pilots  often  lose  control  of  the  situation.  As  an 
example,  a pilot  who  tries  to  escape  will  lose  control  of 
what  is  going  on,  as  he  has  no  longer  radar  contact  with 
the  other  fighters. 
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Wien  a training  session  is  over,  the  pilots  involved 
should  evaluate  the  session.  This  is  typically  a difficult 
process,  partly  because  the  individual  sessions  develop 
in  a complex  way  where  each  pilot  may  have  different 
opinions  on  what  actually  happened,  if  they  are  able  to 
contribute  to  the  situation  at  all.  But  also  because  the 
individual  scenarios  become  indistinguishable  when  the 
pilots  have  returned  to  the  air  base.  As  a result  hereof, 
debriefing  is  deficient  or  non-existing.  Consequently, 
much  value  of  the  training  is  lost.  This  should  be  viewed 
against  the  large  resources  spent  on  keeping  the  fighters 
in  the  air. 

Existing  Solutions  (ACM) 

In  order  to  enhance  the  debriefing  possibilities,  various 
systems  are  available  for  the  pilots  for  recreating  the 
individual  scenarios  that  constituted  the  training. 
Generally  speaking  two  solutions  exists:  A low-cost  and 
an  expensive  solution. 

Low-  Cost  So lution : Video 

The  F-16  fighters  used  by  the  Danish  Air  Force  are 
equipped  with  three  standard  video  cameras,  which 
records  the  Head  Up  Display  (HUD)  and  the  two  Multi 
Function  Displays  (MFD).  The  pilots  can  use  these 
videos  in  a subsequent  debriefing.  Videos  are  excellent 
for  the  initial  scenario  and  evaluation  of  shootings.  In  a 
debriefing  situation,  the  pilots  involved  will  endeavour 
to  recreate  the  individual  scenarios  in  the  training 
session.  If  the  pilot  has  lost  control,  however,  videos  are 
of  little  use  (the  radar  image  may  be  of  no  value). 
Furthermore,  it  is  difficult  and  time-consuming  to 
synchronise  multiple  videotapes  and  ECM  as  well  as  kill 
removal  are  not  covered  by  video  at  all. 

The  Expensive  Solution:  Real-time  ACM  Instrumentation 
(ACMI) 

Real-time  ACMI  covers  the  expensive  and  extensive 
solution  where  the  individual  fighters  that  participate  in 
the  session  downlink  information  in  real-time  to  a 
control  station  on  the  ground.  Via  the  control  station,  the 
individual  scenarios  are  monitored  and  stored  for  later 
debriefing.  The  control  station  may  even  intervene 
during  the  training  session,  either  in  order  to  influence 
the  situation  in  a certain  direction  or  due  to  kill  removal. 

Real-time  ACMI  involves  pod-mounted  electronics 
(GPS,  MUX -BUS  interface  and  data  link)  as  well  as 
antenna  coverage  on  the  ground  and  all  control  facilities 
on  the  ground.  Consequently,  the  solution  is  quite  costly 
in  terms  of  electronic  equipment  and  staffing,  and  ACMI 
will  not  become  a natural  part  of  every  training  session. 
ACMI  must  be  planned  a long  time  in  advance  and  will 
only  be  used  few  times  a year. 

Systematic’s  mission  debriefing  system 

Based  on  informal  discussions  with  both  pilots  from  Air 
Station  Alborg  and  the  Danish  Air  Materiel  Command, 
we  have  developed  a first  generation  model  to  show  the 
possibilities. 


The  first  generation  of  the  debriefing  system  is  an 
autonomous  system  and  does  not  require  any  changes  in 
the  cockpit  or  instrumentation  of  the  aircraft.  The  system 
is  centred  on  a debriefing  facility,  based  to  the  greatest 
possible  extent  on  COTS  hardware  and  software. 

The  debriefing  system  uses  already  existing  information 
from  the  aircraft,  the  Global  Positioning  System  (GPS) 
data,  the  three  videos  (HUD  and  2xMFD),  and  a 
recording  of  pilots’  voice  communication. 

The  HUD,  MFD,  voice  recording,  and  GPS  data  of  the 
individual  aircraft  are  loaded  into  a Personal  Computer 
(PC),  which  synchronises  the  data.  From  the 
synchronised  data  the  PC  constructs  a 2D/3D  synthetic 
world  of  “what  happened”. 

The  three  videotapes  and  the  voice  recording  are  used  to 
give  a detailed  image  of  the  pilots’  actions,  displaying 
what  happened  inside  the  cockpits.  The  GPS  data  from 
all  aircraft  are  loaded  into  a 3D  model  of  the  battle  cube. 
The  3D  model  does,  just  like  a Geographical  Information 
System  (GIS),  contain  a 3D  graphical  model  of  the 
landscape  in  the  battle  cube.  This  3D  model  of  the 
landscape  combined  with  the  aircraft  GPS  data  gives  a 
“Gods  eye  view”  of  the  battle  cube.  The  debriefing 
system  makes  it  possible  to  navigate  around  in  the  battle 
cube.  This  makes  it  possible  to  view  the  scenery  from 
different  perspectives. 

All  aircraft  that  can  provide  the  information  described 
above  can  be  included  in  the  debriefing  session. 
Consequently,  the  system  can  be  used  not  only  by  the 
Royal  Danish  Air  Force’s  F-16  fighter  pilots. 
Furthermore,  a debriefing  system  like  this  can  be  used 
independently  of  the  geographical  location  and  extension 
of  the  individual  training  sessions.  Compared  with  the 
real-time  ACMI  system,  this  provides  an  obvious 
advantage;  the  real-time  ACMI  system  is  not  mobile,  but 
limited  to  the  location  that  is  covered  by  the  antenna 
equipment  of  the  ground  station. 

The  latest  techniques  in  Virtual  Reality  and  3D  have 
been  investigated  in  connection  with  the  construction  of 
the  synthetic  world.  These  areas  undergo  extensive 
research  and  development  within  the  experimenting  field 
of  computer  science,  and  are  consequently  considered  to 
contain  some  of  the  building  blocks  for  the  future 
development  within  HCI  (Human  Computer  Interaction). 
The  debriefing  system  includes  leading  edge 
technologies  within  these  fields.  It  is  our  aim  to  present  a 
system  that  will  delight  and  motivate  the  pilots  to  carry 
out  high-quality  debriefing. 

Development  of  the  debriefing  system 

This  section  is  a brief  description  of  our  approach  to  the 
project.  Based  on  our  interviews  with  potential  users,  a 
retrieval  of  user  requirements  and  a study  of  existing 
COTS  products  and  their  facilities,  we  started  the 
development  process.  Knowing  that  we  had  to  do  with 
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new  technology,  it  was  essential  for  us  to  study  and 
develop  small  prototypes  of  the  different  functionalities 
in  the  system.  We  decided  to  break  the  system  into  three 
main  subsystems,  which  were  to  be  developed  and  tested 
sequentially.  The  initial  aims  were: 

• To  see  if  we  could  develop  3D  graphics  using 
“cheap”  COTS  technology  and  already  available 
data. 

• To  test  the  different  3D  graphical  components/effects 
that  we  wanted  to  make  use  of. 

• To  establish  a 3D- terrain  model,  which  was  suitable 
for  debriefing  purposes. 

• To  establish  a user-friendly  interface  and  the 
framework  from  which  the  debriefing  application 
should  be  prepared  and  presented. 

In  the  following  text  we  describe  each  of  the  initial 
prototypes,  it’s  purpose,  the  method  used  to  develop  it 
and  the  result/experiences  gained. 

Prototype  1 

This  part  resulted  in  a 3D-terrain  model  with  a 
visualisation  of  a number  of  aircraft  including  their 
tracks,  so  that  one  can  get  an  overall  view'  of  the  full 
mission  or  extracted  parts  of  a mission  or  flight. 

Purpose 

• To  get  a 3D-terrain  model  and  to  show  it  on  a PC 
(We  decided  to  get  the  necessary  data  from  the 
Danish  F-16  simulator). 

• To  make  a 3D  visualisation  of  aircraft  (including 
their  historical  tracks). 

• To  create  lively  navigation  and  animation  methods 
(the  aircraft  should  be  able  to  manoeuvre  and 
navigate  in  a realistic  w'ay  so  that  an  aircraft  would 
bank  naturally  wrhen  turning  and  so  forth). 

• To  enable  the  user  to  choose  between  different  angles 
of  view  (e.g.  “God’ s-eye- view”). 

• To  visualise  other  objects  (e.g.  Surface  to  Air  Missile 
sites  with  threat  domes). 

• Portability:  To  be  able  to  port  the  system  between  the 
normal  PC  platform  and  a more  static  SGI  graphical 
supercomputer  with  holobench. 

Method 

In  brief  we  have  had  a very  open  and  innovative 
approach  where  following  main  activities  were  carried 
out: 

• Information  search  on  the  Internet  to  get  components 
and  pieces  of  code,  which  could  be  useful. 

• Selection  of  a portable  visualisation  core  component. 
(Optimizer™  from  Silicon  Graphics). 

• Courses  in  the  use  of  Optimizer™. 

• Prototyping  and  test  using  visualisation  methods  and 
navigation. 

• Get  inspiration  through  the  studies  of  existing  ACMI 
systems. 

• Initial  development  on  PC  — later  ported  to  SGI. 


Results 

These  were  the  results  we  got  from  our  first 
developments: 

• Functionality  to  convert  the  database  from  the  F-16 
MLU  simulator  to  “PC -format”. 

• A prototype  application  showing  a landscape  of  size 
10  x 10  NM. 

• Playback  of  flights.  (Specifically  two  flights  flying 
different  routes.) 

• Possibility  to  see  the  flights  in  a follow-mode  (seen 
from  one  of  the  flights  or  in  a “God’ s-eye -view'”). 

• Portability  between  PC  and  SGI  (holobench). 

• Possibility  to  run  the  application  on  a PC  with  a 
powerful  graphics  card. 

Prototype  2 

The  next  step  was  to  develop  an  application  that  could 
visualise  a complete  geographical  database  and  to  make 
3D  movement  through  the  landscape. 

Purpose 

• To  create  functionality  to  visualise  a complete 
geographical  database  covering  a normal  theatre  of 
operations. 

Method 

• Use  experiences  from  prototype  1 . 

• To  develop  and  implement  efficient  methods  to  get 
and  drop  tiles  of  terrain  in  the  visible  area. 

• To  convert  the  F-16  MLU  simulator  database  to  PC- 
format”. 

Results 

• A prototype  2 application  with  functionality,  which 
in  principle  (if  terrain  data  is  available)  can  show  any 
given  terrain. 

• Geographical  data  enabling  the  system  to  cover 
Denmark  and  Southern  Norway. 

• This  prototype  was  only  developed  for  a PC. 

Prototype  3 

The  third  prototype  is  the  set-up  and  administration  tool, 
developed  on  a Microsoft  Outlook  user  interface. 

Purpose 

• To  obtain  functionality  to  administrate  flights  and 
missions.  (A  flight  is  an  operation/flying  session 
performed  by  one  aircraft  and  a mission  is  a 
combination  of  concurrent  flights  ). 

• To  be  able  to  perform  video  playback. 

• To  synchronise  video  inputs  and  the  3D-terrain 
model. 

• To  present  a graphical  user  interface  (GUI)  for 
debriefing  and  administration  in  a Microsoft  Outlook 
view. 
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Method 

• Standard  components  were  to  be  used 

- Standard  Template  Library  (STL)  from  Silicon 
Graphics. 

- Microsoft  Foundation  Classes  (MFC)  from 
Microsoft. 

- Windows  media  standard  components  for  video 
playback. 

- Microsoft  Access  Database. 

• Use  of  simple  application  development  (Visual  C++, 

6.0). 

• Use  of  well-known  components  for  the  GUI 
(Microsoft  Outlook). 

Results 

• A quad-view  (four  concurrent  views  on  same  screen) 
with  an  intuitive  timeline  that  permits  playback, 
review,  pause  etc. 

• An  intuitive,  easy-to-leam  GUI. 

• Use  of  Windows  standard  functionality  to 
synchronise  video  and  data. 

Integration  to  a first  generation  model 

Before  integration  of  the  three  prototypes  into  the  first 
generation  of  debriefing  system,  we  had  to  solve  some 
minor  problems  that  occurred  during  test  of  prototypes: 

• Geographical  data  are  extensive  and  requires  a 
harddisk  of  at  least  1GB  for  the  database.  We 
improved  our  hardware  to  the  necessary  level. 

• Movements  through  the  3D  terrain  require  loading 
and  initialising  of  huge  amounts  of  data  . Therefore  a 
dual  processor  system  and  a very  fast  harddisk  must 
be  used  to  give  video  and  other  resources  enough 
processing  power. 

• Using  video  and  3D -graphics  in  the  same  session 
creates  performance  problems  Windows  2000 
combined  with  multiple  graphic  cards  solves  the 
problem. 

Once  these  problems  were  solved  we  were  able  to  load 
the  real,  digitised  video  from  F-16  aircraft  and  through 
prototype  3 we  could  initiate,  administrate  and  run  the 
debriefing  system  with  the  introduced  functionality.  The 
result  is  promising  and  after  some  pre-tests  with  real 
users  and  the  necessary  adjustments  and  improvements 
in  functionality  a flexible  system  is  ready  to  be 
implemented  with  operational  fighter  squadrons. 

Use  of  Systematic’s  debriefing  system 

Using  the  Systematic  debriefing  system  is  a 3 -step 
process: 

• Digitisation  of  source  data 

• Mission/flight  set-up 

• Debriefing 

These  processes  are  described  in  the  following. 


Digitisation  of  source  data 

After  completing  a flight,  data  collected  from  the  plane 
must  be  converted  to  formats  suitable  for  computer 
processing.  Analogue  data  must  be  digitised  and  stored 
in  appropriate  formats. 

• Video.  Video  recordings  from  the  HUD  and  MED’s 
must  be  digitised  and  converted  to  “mpgl”  format. 

• Discrete  flight  path  information.  Flight  path 
information  consisting  of  at  least  position  (time, 
latitude,  longitude,  height)  and  optionally  orientation 
(heading,  pitch,  yaw). 

• Event  registrations.  Identification  of  events  that 
occurred  during  the  flight.  These  could  be:  Weapon- 
release,  radar  lock-on  etc. 

• Environment.  Stationary'  and  moving  objects  which 
give  important  input  to  the  flight  debriefing.  This 
could  for  example  be  location  of  a SAM  site. 

Mission/flight  Set-up 

The  purpose  of  the  mission/flight  set-up  phase  is  to 
arrange  the  source  data  into  logical  units  such  as  flights 
and  missions.  For  example,  a flight  is  a container  for  all 
data  relating  to  a flight  including;  name  of  the  pilot, 
identification  of  the  plane,  the  videos  recorded  from  the 
plane  and  the  flight  path  data  from  the  plane. 


Data  is  arranged  in  a hierarchical  structure: 


Figure  1:  Data  structure 


The  different  types  of  data/files  should  be  read  as 

follows: 

• Mission,  A mission  defines  a collection  of  related 
flights.  A debriefing  typically  involves  several 
flights. 

• Flight.  A flight  defines  the  pilot,  the  plane,  a set 
videos  recorded  from  the  plane  and  flight  path 
recording  from  the  plane. 

• Pilot.  Defines  the  characteristics  of  a pilot 

• Plane.  Defines  the  characteristics  of  a plane/aircraft. 
Aircraft  type/model,  visual  representation 
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• Video.  Defines  video  recording  from  a plane  (related 
to  a plane).  Includes  start  and  stop  time  for  the  video. 

• Data.  Defines  flight  paths. 

Debriefing 

A debriefing  is  concentrated  around  a mission. 

The  screen  is  divided  into  five  sections  as  displayed  in 
Figure  2.  Four  of  the  sections  are  dedicated  to  displaying 
video  and/or  the  3D  synthetic  environment.  The 
remaining  section  is  dedicated  to  the  timeline  and 
playback  controls. 


Figure  2:  Division  of  screen  in  debriefing  mode 


The  user  can  make  use  of  following  functionality: 

• 3D  syntetic  environment: 

- God’s-eye  view 

- Follow  mode 

— Free  movement 

• Video-playback: 

- On/off 

- Sound 

• Play-back  control: 

- Play 

- Fast  forward 

- Reverse 

- Slow-motion 

- Single  step 

- Search  (time,  event) 

• Pop-up  time  based  annotations/Attachments  on: 

- Data 

- Audio/ Video 

- Flight 

- Mission 


• Computed  “annotations”: 

- Speed,  Height 

- Distance 

- Radar  coverage 

• Information  layers  (on/off  toggles) 

- Flights 

Combining  commercial  off  the  shelf  (COTS) 
technology  with  military  requirements 

To  reduce  cost  and  improve  the  usability  and  learning 
process.  Through  studies  of  a range  of  commercially 
available  products,  we  have  experienced  that  today’s 
COTS  products  basically  cover  all  given  requirements  to 
a debriefing  system. 

COTS  Hardware 

The  PC  market,  driven  by  the  requirements  set  by  the 
entertainment  industries  “need”  to  produce  more  and 
more  realistic  games,  produces  high-performance 
affordable  systems.  Current  state-of-the-art 
entertainment  PCs  are  capable  of  delivering  the  high 
performance  in  the  areas  essential  to  3D  graphics  and 
video  applications.  The  essential  areas  are: 

• Processing  power  — Fast  processors  are  required  to 
handle  movements  through  the  3D-terrain  model. 
Multiple  processors  are  recommended. 

• Main  storage  Memory  (RAM)  is  essential  to  store 
the  3D-terrain  in  use. 

• Mass  storage  — Harddisk  space  is  needed  to  store 
digitised  videos  and  3D  synthetic  terrain.  Today 
mainstream  harddisks  are  both  fast  and  has  large 
capacities. 

• 3D  graphics  — A 3D  accelerated  graphics  adapter  is 
essential  to  produce  3D  synthetic  environments  at 
suitable  resolution  and  frame  rates.  The 
entertainment  industry  drives  the  need  for  3D 
graphics  performance.  Current  and  next  generation 
consumer  3D  graphics  systems  are  powerful  enough 
to  drive  the  3D  synthetic  environment. 

COTS  Software 

We  have  found  that  most  of  the  necessary  software  for 
the  debriefing  system  is  available  in  different  COTS 
products,  wdiich  can  be  acquired  within  a reasonable 
price  or  directly  downloaded  from  the  Internet.  By  using 
these  products  we  also  make  it  easier  for  the  user  to  learn 
to  use  the  system.  We  decided  early  in  the  project  to  use 
Windows  2000  instead  of  Windows  NT.  The  reason  for 
this  is  that  Windows  2000  can  handle  concurrent  use  of 
video  and  3D-graphics. 

Experiences 

We  have  spent  many  hours  searching  for  relevant 
software  products  on  the  Internet  and  other  places.  We 
have  certainly  gained  benefit  from  these  efforts. 
Generally  speaking  there  is  COTS  technology  available 
— especially  from  the  entertainment  industry  — to 
support  and  develop  a range  of  high-tech,  virtual  reality 
training  systems.  Our  task  has  almost  been  reduced  to 
integration  of  already  well-proven  and  tested  blocks  of 
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software  code.  However,  it  must  be  stressed  out  that  the 
main  challenge  was  to  make  the  individual  products 
work  together. 

Perspective 

The  debriefing  system  has  great  extension  possibilities. 
As  an  example,  the  air  force’s  simulators  could  use  the 
debriefing  system  for  evaluation  of  the  simulation 
training.  By  doing  so,  simulation  and  use  of  the 
debriefing  system  will  become  an  integrated  part  of  the 
general  simulator  training.  Consequently,  the  possibility 
for  evaluating  “what  if’  situations  (situations  where  a 
training  scenario  is  evaluated  against  new  actions)  would 
become  a reality.  An  existing  training  scenario  that  has 
been  practised  and  debriefed  in  the  debriefing  system 
could  provide  input  to  the  simulator.  The  simulator  could 
then  fly  with  the  scenario,  and  what-if  situations  could 
be  simulated  in  order  to  evaluate  the  effect. 

The  interaction  with  other  ground  systems,  such  and  C3 
and  Mission  Planning  Systems,  are  further  areas  to  look 
into.  As  an  example,  the  debriefing  system  could  be  used 
to  build  an  Airspace  Co-ordination  Order  (ACO):  With  a 
“magic  wand”  the  operator  could  guide  and  virtually 
draw  a route  through  the  3D  landscape.  An  F-16  fighter 
could  then  use  the  ACO  generated.  When  the  mission  is 
completed,  the  route  planned  and  carried  out  could  be 
compared  in  the  debriefing  system. 

Another  opportunity  would  be  to  investigate  the 
debriefing  facility  in  an  interaction  with  other  armed 
forces.  As  an  example,  the  Navy’s  air  combat  system 


could  be  investigated  and  evaluated.  One  of  the 
problems  of  the  Navy  in  air  combat  is  finding  the 
optimal  defence  process,  and  the  debriefing  system  may 
turn  out  to  be  useful. 

The  F-16  is  equipped  with  a MUX-BUS  interface. 
Through  this  interface  much  more  information,  e.g. 
weapon-release  can  be  accessed.  Recording  these 
information  and  successively  replay  during  the 
debriefing  will  give  a much  more  detailed  image  of  the 
flight. 

The  opportunities  described  above  are  just  some  areas 
where  it  may  be  possible  to  use  the  debriefing  system. 
When  the  system  is  in  operation,  other  opportunities  are 
likely  to  appear,  and  technology  will  show  us  which. 

Conclusions 

We  have  developed  a mission  debriefing  system  that  in 
principle  covers  the  basic  requirement  and  to  some 
extend  even  exceeds  these  requirements.  No  dedicated 
software  has  been  developed  for  use  in  this  first 
generation  of  the  system.  The  input  to  the  debriefing 
system  is  not  made  especially  for  this  purpose,  but 
already  available  sources  have  been  sufficient  (a 
digitisation  of  the  flight  videos  has  though  been 
necessary).  Available  COTS  software  and  hardware  has 
shown  its  value  for  this  purpose,  which  means  that  the 
main  task  for  us  has  been  to  integrate  already  available 
products  and  input.  As  integration  is  one  of  our 
company’s  main  business  areas,  we  are  able  to  do  this 
quite  fast  and  therefore  within  an  affordable  price. 
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